Stationary Probabilities of Markov Chains with Upper Hessenberg Transition Matrices
نویسندگان
چکیده
In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and efficient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derived, which lead to two alternative algorithms. Numerical analysis of the algorithms is given, which shows the reliability and efficiency of the algorithms. Examples of applications are provided, including results of a discrete time state dependent batch arrive queueing model. The idea used in this paper can be generalized to deal with Markov chains with a more general structure.
منابع مشابه
Stationary Probabilities of Markov Chains
In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and eecient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derived,...
متن کاملApproximations for Markov Chains with Upper Hessenberg Transition Matrices
Abstract We present an approximation for the stationary distribution T of a countably infinite-state Markov chain with transition probability matrix P = (pq) of upper Hessenberg form. Our approximation makes use of an associated upper Hessenberg matrix which is spatially homogeneous one P ( ~ ) except for a finite number of rows obtained by letting p q = pj-i+l, i > N + 1, for some distribution...
متن کاملMarkov Chains with Hybrid Repeating Rows – Upper-hessenberg, Quasi-toeplitz Structure of the Block Transition Probability Matrix
In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.
متن کاملSome Specialized Comparison Techniques for Doubly Stochastic Finite Markov Chains
This paper considers certain comparison techniques involving Markov chains with transition matrices Pa = aI+(1?a)P where P is the transition matrix of a doubly stochastic Markov chain. This paper provides upper bounds for how far the Markov chain with transition matrix Pb is from uniformly distributed after n steps. These upper bounds involve how far the Markov chain with transition matrix Pa i...
متن کاملEvaluation of First and Second Markov Chains Sensitivity and Specificity as Statistical Approach for Prediction of Sequences of Genes in Virus Double Strand DNA Genomes
Growing amount of information on biological sequences has made application of statistical approaches necessary for modeling and estimation of their functions. In this paper, sensitivity and specificity of the first and second Markov chains for prediction of genes was evaluated using the complete double stranded DNA virus. There were two approaches for prediction of each Markov Model parameter,...
متن کامل